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ABSTRACT 

In histopathological image analysis, feature extraction for 
classification is a challenging task due to the diversity 
of histology features suitable for each problem as well 
as presence of rich geometrical structure. In this paper, 
we propose an automatic feature discovery framework for 
extracting discriminative class-specific features and present 
a low-complexity method for classification and disease 
grading in histopathology. Essentially, our Discriminative 
Feature-oriented Dictionary Learning (DFDL) method learns 
class-specific features which are suitable for representing 
samples from the same class while are poorly capable 
of representing samples from other classes. Experiments 
on three challenging real-world image databases: 1) 
histopathological images of intraductal breast lesions, 2) 
mammalian lung images provided by the Animal Diagnostics 
Lab (ADL) at Pennsylvania State University, and 3) brain 
tumor images from The Cancer Genome Atlas (TCGA) 
database, show the significance of DFDL model in a variety 
problems over state-of-the-art methods. 

Index Terms — Histopathological image classification, 
Sparse coding, Dictionary learning, Feature extraction 

1. INTRODUCTION 

Automated histopathological image analysis has recently 
become a significant research problem in medical imaging 
and there is an increasing need for developing quantitative 
image analysis methods as a complement to the effort 
of pathologists in diagnosis process. Consequently, an 
emerging class of problems in medical imaging focuses 
on the the development of computerized frameworks to 
classify histopathological images |[TJ|^. These advanced 
image analysis methods have been developed with purpose 
of relieving the workload on pathologists by sieving out 
obviously diseased and also healthy cases, which allows 
specialists to spend more time on more sophisticated cases. 

In the diagnosis process, pathologists often look for 
problem-specific visual cues in histopathological images in 
order to categorize a tissue image as one of the possible 
categories. Consequently, different customized feature 
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extraction techniques for a variety of problems have been 
developed based on these visual cues m- However, 
a challenging question in medical image analysis is how 
to extract these features. The challenge inherits from 
the richness of geometric structures in tissue imagery and 
the meaningful pathological information at diverse scales. 
Although several methods have been proposed for this crucial 
task, they are mostly exclusively designed for particular 
data sets and are highly dependent on preprocessing steps 
(e.g., color normalization and nuclear segmentation), limiting 
their performance on general histopathology problems. In 
order to mitigate the workload in preprocessing step and to 
develop a more general solution, we propose a dictionary 
learning method relying on a sparse representation-based 
framework that can automatically discover relevant features 
from raw medical images and can be applied to several 
histopathological data sets. 

Sparse representation-based methods are powerful tools 
for image classification pTHT3| . The underlying idea is that 
given a class of images and sufficient collection of bases, a 
test image can be expressed approximately as a sparse linear 
combination of bases. Representing signals using a set of 
learned bases instead of predefined bases, e.g. DCT and 
wavelet bases, has led to state-of-the-art results in various 
applications such as denoising, inpainting and classification 
To achieve a comprehensive set of bases, sparsity 
and task-driven constraints are combined together in several 
ways into optimization problems, which are called Dictionary 
Learning methods. For classification problems, the class- 
specific design of such dictionaries enables class assignment 
via a simple reconstruction error-based metric |[^[^. In 
particular, GDDL |[^ and LC-KSVD p0| enforced the label 
consistency needed between dictionary bases and training 
data for classification. Meanwhile, FDDL fUl encouraged 
coding coefficients to have small intra-class scatter but big 
inter-class scatter. 

Sparsity-based classification schemes have also been pro¬ 
posed for medical applications, recently |22|23l . Specifically, 
Srinivas et al. @0 presented a multi-channel histopatholog¬ 
ical image as a sparse linear combination of training exam¬ 
ples under channel-wise constraints and proposed a residual- 
based classification technique. In addition, Parvin et al. 0 
combined a dictionary learning framework with a Restricted 
Boltzmann Machine to learn sparse features for classification. 
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Being mindful of the challenges of feature extraction 
of histopathological images, we aim to build discriminative 
bases for each class by imposing sparsity constraints on 
minimizing intra-class differences, while simultaneously 
emphasizing inter-class differences. Small intra-class 
differences encourage the comprehensibility of the set of 
learned bases, which has the ability of representing in- 
class samples with only few bases (intra class sparsity). 
Simultaneously, large inter-class differences prevent bases of 
a class from sparsely representing samples from other classes 
(complementary samples). This crucial property of learned 
bases would promote the discrimination ability of the sparse 
code (coefficient vector) for classification. Concretely, given 
a dictionary from a particular class D containing k bases and 
a certain number L <C /c, we define an L-suhspace of D 
as a span of a subset of L bases from D. Our proposed 
Discriminative Feature-oriented Dictionary Learning (DFDL) 
aims to build dictionaries with this key property: any sample 
from a class can be reasonably close to an L-subspace of the 
associated dictionary while a complementary sample is far 
from any L-subspace of that dictionary. Illustration of the 
proposed idea is shown in Fig. 

Contributions: The main contributions of this paper are 
as follows: (1) A dictionary learning method for automatic 
feature discovery in histopathological images to mitigate the 
generally difficult problem of feature extraction in medical 
images. (2) Our framework is a discriminative dictionary 
learning method that emphasizes inter-class differences while 
keeping intra-class differences small, resulting in enhanced 
classification performance. (3) The proposed method is 
applied on three diverse histopathological data sets to show 
the capability of our method in handling a variety of diagnosis 
and grading problems. Extensive experimental results show 
that our method provides outstanding performances even with 
a small number of training images. 

2. DISCRIMINATIVE DICTIONARY LEARNING 
2.1. Notation 

Suppose that we have c classes. The vectorization of a small 
block (or patch) of an imag^ which will be referred as 
a sample, is denoted as a column vector y G For 

i = l,2,...,c, let G and G be 

matrices containing all data samples from class i and its 
complementary samples, respectively. We denote by G 
^dxki dictionary of class i. 

For a code s G we denote by ||s||o the number of its 
non-zeros. The sparsity constraint of s can be formulated as 
||s||o < L with L k. For a matrix S , ||S||o < L means 
that each column of S has no more than L non-zeros. 


^ Currently, we are working on the luminance component only, extension 
of DFDL to multi-channel case will be considered in future. 



Fig. 1: Sparse representation space of a) predefined dictionary, 
e.g. DCT or Wavelet (5 'lo,£o (^^pre) may cover all data), b) learned 
dictionary using in-class samples only, e.g. KSVD p5) or ODL |T6) 
(Din-class) Hiay also cover some complementary samples) and 
c) desired DFDL (5 'l2,£2 (Ddfdl) cover in-class samples only). 

2.2. Problem Formulation 

We aim to build class-specific dictionaries such that 
each reasonably represents samples from class i but is 
poorly capable of representing its complementary samples. 
Concretely, for the learned dictionaries we need: 

^ min IlYi —D^Sdli to be small 

Ni\\Si\\o<u" 

and L min IlYi —DiSjllp to be large 

Ni II Si II < Li 

where Li controls the sparsity level and || • denotes the 
Frobenius norm. For simplicity, from now on, we consider 
only one class and drop the class index in each notion, i.e., 
using Y, D, S, S, N, TV, L instead of Y^, D^, S^, S^, Ni,Ni 
and Li. Based on the argument above, we formulate the 
optimization problem for each dictionary: 

D*=argminf^ min IIY —DSIIo 

D VW ||S||o<L 

--t min ||Y-DS||i') (1) 

A^I|S||o<l" "v 

where p is a positive regularization parameter. The first 
term in the above optimization problem minimizes intra¬ 
class differences and the second term emphasizes inter-class 
differences. By solving the above problem, we can find the 
appropriate dictionaries as we desire. 

In the same manner with SRC a new patch y is 
classified as follows. Firstly, the sparse codes s are calculated 
via /i-norm minimization: 

s = argmm{||y - UtotaMll + A||s||i} 

where 'Dtotai = [Di, D 2 ,..., T>c] is the collection of all 
dictionaries and A is a scalar constant. Secondly, the identity 
of y is determined as: arg min {(^i(y)} where 

l<i<c 

Si{y) = ||y-Di(5i(s)|| 

and (5i(s) is part of s associated with class i. 




Data: Y, Y: collection of all in-class samples and 
complementary samples. 
k\ number of learned bases. 
p\ regularization parameter. L\ sparsity level 
Result: D: dictionary 

1. Initializing D by randomly picking k columns of Y 
while not converged do 

2. Fix D and update S, S by solving Problem 

3. Fix S, S, calculate: 

if F is not PSD then 
I p 0.9p; go to 3.; 

end 

4. Update D by solving Problem 0; 

end 




Algorithm 1: DFDL for sparse representation-based 
classification 


2.3. Proposed solution 

We use an iterative method to find the optimal solution for 
problem 0. Specifically, the process is iterative by fixing D 
while optimizing S, S and vice versa. 

At sparse coding step, S*, S* can be found by solving: 

S* = arg min IIY—DSII S* = arg min IIY—DSII ^ 

l|S||o<I^" " l|S||o<L 


With the same dictionary, these two sparse coding 
problems can be combined into the following one: 


S* = arg min 

l|S||o<I^ 


Y-DS 


( 2 ) 


with Y = [Y, Y] being the matrix of all training samples and 
S = [S, S]. This sparse coding problem can be solved by 
OMP method p4| using SPAMS toolbox pS] . 

For the bases update stage, D* is found by solving: 

D* = argmino {i||Y - DS||| - ^||Y - DS|||.} (3) 

= argmiuD { — 2trace(ED^) + trace(DFD^)} (4) 

using the method of block coordinate descent with a warm 
start to update bases one by one |T^ . We have used the 
equation ||M|||. = trace(MM^) to derive 0 from ^ and 

denoted E = ^YS^ - -^YS^; F = - -^SS^. 

N N N N 

In order to make the optimization problem tractable and 

solve it efficiently, we need one more requirement for p 

to ensure that — 2trace (ED+ trace (D^DF) is a convex 

function with respect to D, or in other words, the symmetric 

matrix F need to be positive semidefinite (PSD). If we let 

Ai(M) < A 2 (M) < • • • < Amax(M) be (real) eigenvalues 

of a symmetric matrix M, the PSD constraint of F is 



(a) UDH (b) Healthy Lung (c) Not MVP 



(d) DCIS (e) Inflamed Lung (f) MVP 

Fig. 2: Samples from three data sets. Column 1: IBL data set, 
column 2: ADL data set, column 3: TCGA data set 


equivalent to the non-negativity constraint of Ai(F). Using 
Weyl’s inequalities, we can get lower bound for Ai(F):Ao = 
iAi(SS^) - ^A„,a.,(SS^) < Ai(F). As a result, if p is 
guaranteed to be small enough such that Aq > 0, then F is 
PSD. In fact, this problem is unstable and difficult to track 
since S, S depend on D. We propose a practical solution for 
this difficulty as follows. First, we start with a small value 
of p and check if F is PSD at each iteration. If so, p remains 
unchanged; otherwise, p would be assigned to a smaller value, 
say, 0.9p. In our experiments, p = 0.001 gives good results. 
Our DFDL method is summarized in Algorithmic 

3. EXPERIMENTAL RESULTS 

In this section, we present the experimental results of 
applying DFDL to three diverse histopathological image 
databases (sample images are shown in Fig. 0 ) and 
compare our results with those using SVM in conjunction 
with a collection of state-of-the-art histopathology features 
from WND-CHARM Q (will be referred as WND-CHARM 
method), SRC | [TT| , LA-SHIRC (^1 and other DL methods 
(LC-KSVD and FDDL ||^). In each experiment, 
10000 20-by-20 patches are randomly extracted from training 
images for each class. Each Dictionary Learning method 
learns the same number of bases, say 500, per class. 

IBL data set contains images which belong to either of 
two well-defined classes: usual ductal hyperplasia (UDH) and 
ductal carcinoma in situ (DCIS). Ground truth class labels for 
the images are assigned manually by the pathologists. A total 
of 40 patient cases - 20 well-defined DCIS and 20 UDH - are 
identified for experiments in the manner described in | |26| . 
Each case contains a number of regions of interest (Rols), 
and we have chosen a total of 120 images (Rols), consisting 
of a randomly selected set of 20 images for training and the 
remaining 100 Rols for test. Images are downsampled for 
computational reduction purpose such that size of a cell is 












Table 1: Classification accuracies: IBL and ADL Lung 
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Fig. 3: Examples of learned bases from 
DCIS dictionaries. 


UDH and 


(b) 


around 20-by-20 (pixel). In classification step, an image is 
decomposed into non-overlapping patches and it is classified 
as Healthy if proportion of classified-as-healthy patches is 
higher than a threshold. In order to mitigate the issue of 
well-chosen training sets, we perform 10 different trials of 
each experiment with an arbitrary choice of training images. 
All results reported next are the average of the classification 
accuracies over 10 trials. 

Learned bases corresponding to the two classes are 
visualized in Fig. The average classification accuracy 
for each method is shown in Table [T] It is evident from the 
table that DFDL outperforms others, offering a classification 
accuracy of nearly 100 percent in recognizing DCIS and just 
under 97 percent in UDH. It means that by using DFDL 
method, the probability of miss is extremely low while that 
of false alarm is kept at a low level. In order to illustrate 
the efficiency of DFDL, we keep number of training patches 
and learned bases but reduce the number of training images 
by half. Noticeably, DFDL still shows outstanding results 
compared to the other methods. 

ADL-Lung data set: This database contains bovine 
histopathology images of lung acquired by pathologists at 
the Animal Diagnostics Lab, Pennsylvania State University. 
These images are scanned using a whole slide digital scanner 
at 40x optical magnification and are of size 4000 x 3000 
pixels. For the purpose of computational speed-up, all images 
are downsampled to 400 x 300 pixels in an aliasing-free 
manner. This database consists of images from two classes: 
healthy and infiammatory. Each class has 150 images from 
which 40 images are chosen for training, the remaining ones 
are used for testing. The averaging experiment results over 
10 trials for different methods are presented in Table 
Apparently, SHIRC and LC-KSVD are moderately suitable 
to detect infiammatory while WND-CHARM only provides 
a fairly high-performance in recognizing healthy organs. In 
contrast, DFDL offers the best accuracy in both detecting 
Healthy and Inflamed organs with more than 92 percent in 
the former and over 94 percent in the latter. 


Method 

IBL (%) 

ADL Lung (%) 

UDH 

DCIS 

Healthy 

Inflamed 

WND-CHARM 1^ 

86.36 

90.91 

88.75 

62.38 

SRCf^ 

68.00 

56.00 

72.50 

75.83 

SHIRCl^ 

93.33 

90.00 

75.00 

85.00 

LC-KSVD^ 

58.73 

54.13 

77.03 

83.93 

fddlE^ 

84.62 

91.84 

- 

- 

DFDL^ 

96.67 

98.63 

86.10 

91.28 

DFDL 

96.94 

99.74 

92.34 

94.24 


reduce number of training samples by half. 


Table 2: Confusion matrix: MVP and Not MVP 


Class 

Not MVP(%) 

MVP(%) 

Method 

Not MVP 

66.98 

33.02 

WND-CHARM Q 

71,70 

28.30 

DFDL 

MVP 

12.50 

87.50 

WND-CHARM Q 

09.37 

90,63 

DFDL 


TCGA data set: In this section, we present experimental 
results on the brain cancer histopathological images obtained 
from TCGA database | [27| provided by the National Institute 
of Health. One important indicator of a high grade glioma is 
presence of Micro Vascular Proliferation (MVP). Essentially 
MVP is presence of proliferation of hypertrophic endothelial 
cells in the tissue. An example of a tissue containing MVP 
regions is illustrated in Fig. In this paper, we applied 
our method to find MVP regions with a slight modification 
to the decision procedure. This modification is crucial to 
obtain desirable performance using our algorithm since MVP 
detection is an inherently more difficult problem because of 
the complexity of cell structure and morphological features 
of the cells in MVP regions. 

Unlike classifying images in IBL and ADL Lung data 
set which are distinguishable by researching small regions, 
MVP detection requires more effort because an MVP region 
might be surrounded by tumor cells which are actually low 
grade. We define a patch as MVP if it lies entirely within 
an MVP region and as Not MVP otherwise. A new image 
is divided into non-overlapping patches and it is classified 
as MVP only if it contains a sufficiently large number of 
neighboring classified-as-MVP patches. 

We use a total of 178 images (resolution 1800 x 1800) 
from the TCGA, 52 for MVP and 126 for not MVP and 20 
images are randomly selected from each class for training. 
We manually extracted MVP and Not MVP regions from 
training images and randomly extract training patches from 
these regions for learning. Experimental results for DFDL 
and WND-CHARM are presented in Table The Table also 
shows promising performance of DFDL in detecting MVP 
with a accuracy over three percent more than those for state- 
of-the-art WND-CHARM features. 
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